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Four experiments' were conducted to determine if 
giving students specific instructions about the, nature of the textual 
material and the finil test is as effective as giving them practice 
questions in learning. from text. Subjects for experiments 1 and 2 
wexe 80 San Diego State University students in an introductory 
psychology- course. Subjects for experiments 3 and 4 were 279 Navy 
enlisted personnel. In all experiments, subjects were randomly 
assigned to one of four groups: a readronly control group, a practice 
questions group, an instructions group, and a practice questions plus 
instructions, group. The four experiments (1) compared the effect of 
instructions and practice questions on learning verbatim factual 

information; (2) replicajted the procedures Af_the__first-,.-but^^ 

paxaphrased-c^omprehehsion questions instead of verbatim questions; 
(3) examined the effects of instructions and practice questions on 
learning as a conceptual" classification task from the text; and (4) 
replicated the third experiment with revised instructions. Results 
indicated that instructions can be as effective or more effective 
than practice questions in learning from text, that instructions 
control/focus student processing and attention as well as do practice 
questions, and that the best instruct ional strategy is a combination 
of instructions and practice questions. .(Instructions used in the 
experiment are appended.) (YLB) 
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FOREWORD 



This- effort was conducted under project ZR000-0i-0'f2-06-01.0i (Enhancement of 
Information Acquisition). TJ:ie purpose of this project is to provide a greater understand- 
ing of the effects of instructional strategies employing instructions and practice questions 
on student study behaviors. 

This report describes a series of experiments conducted to investigate the effects of 
giving students explicit instructions on learning from text. Results of the research are 
intended for tise by the Naval Education and Training Command. 



.3AMES F. KELLY, 3R. 3AMES W TWEEDI3aLE 

Commanding Officer Technical Director 
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SUMMARY 

V 

Pro blem 

' Many variables have been shown to facilitate learning from text; for example, giving 
students advance organizers, practice questions, or behavioral objectives; having them 
take notes or summarize, etc. However, little information is available as to how these 
variables affect/control student study behavior; that is, why the effects occur. 

Objective 

The objective of the current effort was to determine if giving students specific 
• instructions about the hature of the instructional materials and the final test items is as 
effective or more effective than giving them practice questions in focusing their 
attention while learning from text. 

Approach ' ' 

Four experiments were conducted that corttrasted giving students practice questions 
with giving them specific instructions about the final test. These experiments were based 
.on the idea that students are actively attempting to discover what is important while they 
are reading text and that providing them with information on whaf to study' enhances 
performance. Results of previouslstudies in which learners have been given instructions 
suggest that the content of instructions can direct learner attention to impor'tant aspects 
of the materials they are learning and make them more effective learners. Although 
practice questions can also focus learner attention, they may be less effective because 
students must infer what is important from the content of the questions. 

In all experiments, subjectr were randomly assigned to one of four experimental 
groups: (r) a read-only control group, (2) a practice-questions group, (3) an instructions 
group, and W a practice-questions-plus-instruction^ group's. In all experiments, tests 
were developed that included "practice" and "incidental" questions. The former were 
"practice" only for the two practice-question groups, who answered them after reading 
their "experimental materials. The read-only control and instructions groups had not seen 
them before. 

' Results 

Experiment 1 • 

The first experiment compared the effect of instructions and practice questions on 
learning verbatim factual information. Results showed that subjects in the practice- 
questions groups and in the instructions.^\jp performed significantly better than did 
those in the confol group. - Incidental questions were answered somewhat better hy 
"instruction" subjects than by "practice-question" subjects. 

" Experiment 2 

The second experiment replicated the procedures of the first. However, the practice" 
and incidental questions were derived by paraphrasing factual information fr^-m the text, 
and instructions on how to study for these questions were given to the two instructions 
groups. All groups performed significantly better than did the control group. The two 
practice-questions groups answered the practice questions on the final test somewhat 
better than did the instruction group. 
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Experiment 3 > 

The third study replicated the procedures of the first two. In this case, the task 
involved learning to classify instances of several different categories. Results showed 
that the practice-questions groups performed better than did the control group on both 
practice and incidental test questions that required them to classify call signs. The 
results indicate that-, by themselves, instructions about what a classification task is and 
how best to process the information were not as effective as practice questions and 
student performance did not differ from that of the read-only control group. Protocols 
revealed that the instructions had not been well understood by the subjects. 

Experiment ^ ^ 

The fourtir~exper-iment replicated the third with revised instructions based on the 
protocols from experiment 3. Results show that the instructions and {jractice-questions 
groups did equally well on practice and incidental questions requiring classification of call 
signs. All groups did better than the control group. 

Conclusions . " - . . % 

The results of these studies indicate that (1) instructions can be as effective and, in 
some Ckses, more effective than practice questions^in facilitating learning from text, (2) 
instructions, in some cases, more directly control/focus student processing and attention 
than do practice questions, and (3) a combination of instructions and practice questions is 
likely to be most effective. More research is needed to investigate how instructions 
actually work to change . or modify the- student's studying behaviors. This can be done by 
looking more closely at the processes the student employs while reading the instructions 
and studying the materials. Knowing more about what the student is doing and how the 
instructions affect his/her study behavior, will enable instructional designers to be more 
precise in their prescriptions for writing instructions and for determining what specific 
kinds of instructions are most appropriate. 

R ecom m endations <> 

1. Students shoul 1 be given both explicit instructions and questions during both 
classroom and individualized instnuctional situations. 

2. It is recommended thaf instructions provided in the appendix be used with the 
appropriate instructional content. 
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♦ . INTRODUCTION 
Problem ' >»• ' 



■ Many variables have been shown to faciliiate learning from text; for example, giving 
students advance organizers, practice questions, or behavioral °bjectives; havmg students 
take noles or summarize, .etc. However, little information is available as to how these 
variables affect/control student study behavior; that is, why the effects occur. 

Background 

Many experiments have investigated. the effects of practice questions on learning. 
Anderson and Biddle (1975) and Rickards (1979) provide good summary reView^ In the 
typical study, the performance of one or more groups receiving practice questions is 
compared to that of a control .group receiving no questions. Subjects in all groups are 
given a reading passage. They are usually not allowed to refer back to the reading 
Srial to ansler. thi practice questions; rather, they must rely solely on memory to 
answer the questions. After completing the passage, all subjects are given a test to assess 
the effects of the practice questions. • ■ * 

Some of the more interesting findings occur in studies of practice questions that 
follow text passages (practice post-questions) as opposed to studies of questions that, 
precede text%asslges%ractice%re-questions). Practice P°^t--q"^^^;°"^^?" 
fypes of effects: (1) a "direct effect"; that is, .post-question groups ^^'l^^'^J^^fJ'^'' 
read-only control group on final test questions that are informationally similar or 
dJntical \o the prictice post-questions, and (2) an "indirect effect^' where subject 
receiving post-que'stions perform' better than control subjects - .ff^^— 
are unrelated or incidental to the practice questions (Anderson & Biddle, 197^^°™^^ 
& Bisbicos 1967). The indirect effect is important because it shows that practice 
questions that follow sections of instructional materials can help the student learn 
information other than that covered in the questions. ' 

Many investigators have hypothesized that the indTrect effect of.C"'-^, ''f j^".^ 
practice questions focus th<?>,student's attention on the type of question (e.g., factual, 
application) and/or type of information (main idea? or details such as date^ places^ 
names etc.) that will be included on the final, test (Anderson & Biddle, 1975; Rothkopt, 
1966) Focusine is hypothesized to occur in two ways. First, the practice post-questions 
i!n irt ?he stVdent'as to the type of information to study. This is called the "forward 
effect," and results in increased attention to the text following the questions (McGaw & 
SVoTel'ueschen, 1972). Second, if it is assumed that students mentally T^^iew what they 
have iust read to answer practice post-question, they might alsp review, and perhaps 
learn,^ material in the topfco-spatial neighborhood of the fi ectly questioned m^^^^^^^^^^ 
.This is called- the "backward review effect" (McGaw & Grotelueschen, l?72j Rickards & 
DiVesta, 197'f; Rothkopf & Billington, 197'f). - • 

The notion that questions focus attention follows from.the assumption that students 
iPL a learning situation form hypotheses about what is important to study, and that 
practice pisf-questions facilitate this process by focusing students' study behaviors and 
'fnformation processing (cf. Andre, 1979), Thus, P^-^^tice post-questions have a 
"rnShemagenic" effect be9au3e students use them to "figure out" what, the final, test will 
be like (Rothkopf, 1966; Rothkopf & Bisbicos, 1967). . 

Brown, Campione, and Day (1981) support the ideas that students actively attempt to 
discover what is important while'reading text and that providing information about what 



to study enhances performance. They studied $he effects of instructions and training 
designed to Improve students' self-control and self-awareness of their own learning 
processes. They found that students learned mOre effectively, when they were provided 
w.lh instructions and training on how "to study pertain types of material and on. how to 
monitor and control their' study activities than when they did not ' receive, instructions. 
Brown et al. (1981) concluded that students cannot help but become more effective 
learners if they are made aware. of (1) ba'sic strategies for reading *and remembering, (2^ 
simple rules of text construction, (3) differing demands of a .variety of tests to whielr their 
information may be put, and W the importance of activating any background knowledge 
that they may have.' " - . . ' , * 

k • 

Further support for the, hypothesis comes from Mayer and Bromage (1980) in the^r 
studies involving advance organizers. They found that a group that received a conceptuaP 
advance organizer bfefore reading the text scored higher in recall of conceptual ideas, 
recalled more incidental material, and made more novel inferences on a recaU test than 
did a group that received the advance organizer after reading the text. In terms of the^ 
present discussion, the advance organizer provided the students ^ith "cluesV ^bout the 
nature of the fi»§l test and focused'tlieir study strategies and behaviors. 

Duell j(197^) provides 'some support for the hypothesis that giving* students informa- 
tion about what is important ta study facilitatesxlearning. She provided ,some students 
with detailed behavioral objectives that directed them to learn concepts, names^ dates, 
and definitions presented in the text; and Other students, with a- nonbehavioral objective 
that stated that they would be given a multiple-choice test after the reading. After all 
students had read- the text passage and completed the test,' they were asked -to classify 
each tesV item as important or unimportant. The items classified by all subjects as 
unimportant tested recall of names and dates, and those classified as important tested 
recall of concepts and definitions. Dueli found that students who. receiyed detailed 
behavioral objectives performed significantly better than did students who receiyed a 
nonbehavioral objective on unimportant questions but not on important questions. She 
* concluded that .other strategies for focusing student^ study , behavior may operate in a 
similar manner (i.e., by setting-off information needed, when the learner is .tested and 
ensuring that the learner is aware of this relationship). 

Finally, Rothkopf (1966), in an investigation of practice questions, compared the 
performance of a group w^o received, specially prepared care-inducing instructions with 
that of practice-question groups and ^ control group, the instructions included state- 
ments that. the reading material contained much detailed factual information and that the 
text should be read carefully and slowly. The results, which .showed" that the instructions 
group performed bettep-4han did a control group, support the notion that instructions c^n 
facilitate performance. ^ ^ . 

Objective * ' . 

The objective of the present efforf was to 'determine Jf giving students specific 
instructions about the nature of the instructional* materials arid the final test items is as 
effective or more effective, than .giving them practice questions jn focusing their 
attention while learning from text. 

APPROACH 

t " ' • • 

In the present effort, four experiments were conducted that contrasted giving 
students practice question? with giving them specific instructions about the final test. 
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The experiments are based on the idea, that students are actively attemptinglto discover 
^ what is important while they are reading text and that providing information about what 

' ^ to study enhances performance. This enhancement should occur bepause^instructions are 
"explicit," while practice questions are "implicit." By ^^explicit," it is meant that the 
instructions are rpt vague or ambiguous; father, they provide the student with essential , 
information about the nature of the test and hov/ best to process the infornrtatipn. By 
"implicit," Tt^is meant that the student must induce what the test will be like from the 
type of questions asked during instruction. If the student forms a. wrong hypothesis or, 
concludes that only the information covered in the questions is important, performance on 
incidental questions may not [improve. This is hypothesized to be the reason that 
incidental effects are not founk with practice preque]stions (Anderson & Biddle, 1975). On 

' the other 'hand, if the student can b*e given explicit information about what the test will 
be. like before- 'Studying, and about how best to "process" the information that is to be 
r learned, test performance should improve. 

^ Rtsjjlt^' of receot stOdies' (e.g.. Brown et al., 1981) on the performance of students, 
who have received instructions suggest that the conterit of instructions can direct their 
attention to important aspects, of the materials they are learning and make them more 
' • effective learners. ' Although practice questions 'tan also focus student attention, they ^ 
may be less efficient because students must infer what is important from the content of 
the questions. Therefore, thd problem is to determine if giving studgnts explicit 
instructions is as effective or more. effective as.an instructionSl^strategy than giving them 
questions. . • " ' - V ^ 

EXPERIMENT 1 . 

Th$ first experiment compared the effects of instruetions and practice q^iestions on 
^ * learning verbatim factual information from text. " ' , 

Method ' ' ' ' ' ^ . ' 

Subjects and Materials ^ , . ' 

. Subjects, 80 San Diego State University students enrolled .in a class on Introductory 
Psychology, were randomly assigned to ^f our groups of 20 each:. (0 a read-only control 
group, (2) a practice-questions group, (3) an instructions group, "^and (^) a practice- 
questions-plus-ihstructions group. The 1552-word experimental passage concerned 
animals and minerals found in the, ocean, and was taken from Rachel Carsori^s (1951) The 
Sea Around Us . It was composed of a series of factual segments and was divideid into' 1*0 ' 
. sections. ' - « ^ - . 

A test was developed that included 20 factual questions, written by taking words and 
phrases verbatim from the 'passage. There were two questions from eaqh^ section: a 
"practice" question, which would be practice, only ^f or the practice-questions and the 
practice-questions-plus-instructions groups, and an "incidental" qCtestion,- which would ngt , 
be seen by any of the groups until they took the test.,. Se.ts of atceptable answers w^re 
generated for each question for use as the scoring key. In tfTe test, the 20 questions were 
arranged randomly "so that the.order presented did not correspond to the order of the 10 
sections. ' ' ^ . - 

The .practice. and incidental questions wdre not. counterbalanced. Instead, they were 
pretested on a sample of 20 subjects, who read the material and took the test. The test 
Items were then equated for^ difficulty and randomly divided into practice and incidental 
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questions. The mean scores of pretest subjects for practice and incidentafqbestions were 
5.f9 and, 5.57 respectively. 

Procedure 

As the subjects entered the testing room, they were randomly assigned to one of the 
experimental conditions. After they had worked through the appropriate materials at 
their own pace, they took the 20-question short-answer test that required them to recall 
verbatim factual information presented in the passage. Treatments of the g.oups are 
described below: 

1. ' The read-only control group read the 10 sections in the passage and took the 20- 
question test. 

- 2. Aiter subjects in the practice-questions group had read each section of the 
passage, they were given the practice question written for that section and instructed to 
answer it without referring back to the materials. Subjects were monitored to prevent 
them from referring back to the passage and to see that they followed the instructions. 
(No subjects were eliminated for faUure to foUow instructions.) After subjects had 
completed all of the sections, they took the test,. which included the 10 practice questions 
they had ^ready answered and the 10 incidental questions they had not seen before. 

3". The instructions group was given instructions about the final test. These 
instructions, which were designed to be more explicit and detailed than were Rothkopf s 
(1966) hortatory instcUctions, provided specific information regarding the nature ot the 
test and the type of information to be tested, as well as general instructions about how to 
study the information to be learned. (The instructions are provided in the appendix.) 
Specifically, subjects were shown a sample passage that was not related to the 
experimental pa'Mge, and then were given two sample questions drawn from it. They 
were told that the sample questions required simple recall of factual information 
presented in the passage. Subjects were then given the experimental passage and 
instructed to-5tudy it so that they would be able to answer factual questions. After they 
had finishecf each section of the experimental passage, they were given the following 
reminder: "Remember, you will be tested on the factual material in the text you are 
reading. Stop briefly and think about what you have just read, then proceed to the next 
page of the. text." When they had completed all of the sections, they took the 20-item 
test. 

k The practice-questions-plus-instructions group received the sample passage and 
the sample questions. Also, after they had read each section in the experimental passage, 
they were given the practice question, followed by the study reminder. They then took 
the 20-item test, which included the 10 practice questions they had seen before. 

Analyses 

1. An analysis of variance (ANOVA) comparing the groups on their performance on 
the practice and incidental questions in the final test was computed. 

2. Group mean scores on practice and incidental questions on the final test were 
compared using a post hoc Newman-Keuls test. 

3. Correlations between group mean scores on the incidental and practice test 
questions w^ere computed. 

X o 
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Hypotheses 

It was hypothesized that: 

I. If giving students practice post- questions results in mathemagenic effects be-; 
cause such questions help them to anticipate what the final test will be like, then gwing 
them instructions about what the test wUl be like should a so be effective Specif ically if 
giving students practice post-questions causes them to focus more on specific content 1 
than on the rest of the material, then stuo^-nrs receiving instructions would perform 
better than would students receiving practice post-questions. Although both practice and 
incidental questions were new for the students given instructions (and for s udents in the 
■ control group), the instructions had been developed to apply, in general, to factual 
questions. 

2 Students receiving practice post- questions should perform better on the practice 
questions when they are repeated on the test than those not receiving practice post- 
questions. 

3. Students that receive both instructions and practice post-questions should per- 
form best on the practice and incidental test items. 

Results and Discussion 

ANOVA results showed that there was a significant main effect for groups (F 
(3 76) - 6 99 P < .001, MSE = 5.22) and for test question type (practice or incidental) {F 
1 76 = 9.96, 1 < .002 aTlTell as a significant test-question-type-by-groups mteracUon^ 
3 76 - 5 08 p < .002 MSE = 1.50). Table 1 presents means and standard deviations (SDsl 
or each groSp on t'hTTumber of incidental and practice test ^^^fons answered 
correctly. N<rte that there was little difference in performance on the practice and 
JncTdentai test items for the read-only control and instructions groups. This confirms that 
the practice and incidental item sets were equally difficult. 

Table 1 

Group Mean Scores and Standard Deviations on Practice 
and Incidental Test Questions— Experiment 1 



Test Question Type 



Practice^ 



Incidental 



Mean*' 



SD 



Meant* 



^Questions were practice only for the practice-questions groups. 
''Based on a maximum score of 10. 



SD 



Total 



Read-only control 

Instructions 

Practice-questions 


5.85 
6.85 
7.35 


1.72 
1.38 
1.75 


5.55 . 

7.30 

6.30 


2.28 
1.52 
2.27 


11. *0 
1*.15 
13.65 


P rac t ice-que stions-plus- 
instructions 


8.80 


1.23 


7.25 


2.12 


16.05 
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The Newman-Keuls comparison of group means for practice questions on the final 
test revealed that the performance of the practice-questions-plus-instructions group was 
significantly higher than that of all other groups (£ < .05). Apparently, the combination of 
instructions and practice post-questions has a cumulative Effect on performance. 
Furthermore, the performance of the practice-questions and instructions groups differed 
significantly from that of the read-only control group < .05). This finding illustrates 
the "direct effect" of practice post-questions and shows that instructions had a similar 
effect. The finding that instructions were as effective as practice post-questions on this 
measure was contrary to expectations. It could be that instructions are more effective 
than initially hypothesized or that practice questions are less effective because of their 
Implicit nature. The findings for the practice-question-plus-instructions group favor the 
latter explanation. ■> i 

The Newman-Keuls comparison of group means for incidental questions on the final 
test showed that the performance of the practice-questions-plus-instructions and the 
instructions groups differed from that of the read-only control group < .05), but not 
from each other. In addition, the performance of the practice-questions group did n6t 
differ from that of the read-only control group. This finding is inconsistent with that of 
Rothkopf and Bisbicos (1967), who found that restricted categories of questions (e.g., 
names, scientific or technical words, distance, etc.) facilitated learning of restricted 
categories of text content. However, the practice questions used in this study required 
recall of general factual information rather than recall of restricted categories. Although 
the difference between the performance of the practice-questions and the instructions 
groups was only marginally significant (^ < .10), it is supported by the significant group- 
by-test-question-type interaction. 

Finally, the correlations between performance on the incidental and practice test 
questions for the read-only control, instructions, practice-questions, and practice-ques- 
tions-plus-instructions groups were .51, .e^t, .55, and .66 respectively, all significant at the 
.05 level. These data indicate that performance on the practice and incidental test 
questions was significantly related in all treatment groups. 

The results of this experiment support the .hypothesis that giving students instructions 
about the final test can be as effective as, and, in some cases, more effective than, 
practice questions in focusing attention and study behaviors on verbatim factual material. 

EXPEEUMENT 2 

The second experiment was a replication and extension of the first. However,' 
paraphrased-comprehension questions (Anderson, 1972) were used instead of verbatim 
questions because it has been shown that such questions given during instruction can 
improve test performance (Andre & Sola, 1976; Andre & Womack, 1978). This finding 
suggests that leading students to process what they read produces greater comprehension 
of and learning from instructional materials. Because paraphrase questioning has been 
found to be a useful instructional strategy, the intent of this second experiment Was to 
determine whether explicit instructions could be written to enable students to prepare 
effectively for paraphrase test questions; that is, could giving students instructions plone 
be as effective as giving them paraphrased practice post-questions? 
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Method 



Subjects and Materials 

Subjects, 87 Navy enlisted men, were randomly assigned to four groups: a read-only 
control group, a practice-questions group, an instructions group, and a practice-questions- 
plus-instructions group. The 2281-word experimental passage concerned the liistory of 
submarine warfare, and was taken from the Naval Orientation Manual (NAVPERS 
16138.F). It was divided into 13 sections. 

A test was developed that included 26 questions written by paraphrasing statements 
in the reading passage according to Anderson's (1972) rules and techniques. They required 
the student to recall factual information presented in the materials and contained no 
repetition of key words or phrases that could act as memory cues. As before, there were 
two questions— practice and incidental— from each section, which were counterbalanced 
The first section of the passage and the two paraphrased questions written for it are 
illustrated in Figure 1. 



Submarine Warfare 

The first submarine to enter combat was invented in 1776 by an American, David 
Bushnell. Revolutionary in every sense of the word, BushnelPs Turtle made a submerged 
attack on a British warship in New York Harbor. Operated by a hand-worked propeller, 
her tanks flooded by a valve and emptied by a hand pump, the Tuftie encountered 
overpowering difficulties and her maiden foray was ridiculed as a failure. Not until 
after the war was it known that Bushnell's undersea boat worried the British into 
moving their blockading warships from New York Harbor to the outer bay. 

1. The effect of the Turtle's presence on British combat strategy was to remove 
blockading warships . ^ 

2. The primary reason for the Turtle's failure to complete its mission was unreliable 
submersion equipment . 
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Figure 1. Example passage and two paraphrased questions. 

Procedure 

The procedure was the same as that used in the first experiment except that the 
directions given to the instructions group differed from those given in the first 
experiment. This time, subjects in the instructions group were given a 26'f-word sample 
passage and w.ere told that it was similar to the experimental materials they were about 
to read and that test questions would use different words that described the same ideas. 
Tiiey were told to study the experimental passage so that tKey could understand the main 
facts presented. (The instructions are provided in the appendix.) After xhey had 
completed each of the 13 sections, they were given the following study reminder: 
"Remember, you will be tested on factual information found in the text. Because the 
wording in the questions will be different, you should study so that you understand the 
main facts presented in the passage." 

it? 



For each subject, the following data were collected: (1) time to complete the study 
materials, (2) time to complete the test, andj3) scores on the practice question and 
incidental questions of the final test. The^short-answer responses were scored by four 
judges, with questionable responses being resolved by group consensus. 

Analyses 

1. ANOVAs were computed comparing the groups on (a) ,the time required to 
complete the study materials, (b) the time required to complete the test, and (c) scores 
obtained on the practice and incidental test questions. 

2. Group mean scores on practice and incidental questions on the final test were 
compared using a post hoc Newman-Keuls test. 

3. Correlations between group mean scores on the incidental and practice test 
questions were computed. , 

Results and Discussion 

The ANOVA comparing groups on time required to complete the experimental 
materials showed a singifrcant main effect for groups (F»(3,83) = 16.92, £< .001). The 
average time needed to complete the materials for the read-only control, instructions, 
practice-questions, and practice-questions-plus-instructions groups were 18.0, 18.6, ^f .S, 
and 28.2 minutes respectively. The ANOVA comparing groups on time to complete the 
final test showed no significant differences among groups. It is clear that the 
introduction of practice post-questions lengthens study time but not test time. 

The ANOVA comparing groups on the number of correctly answered practice and 
incidental final test questions showed significant main effects for group (F.(3,83) = 6.6f , 
p < .001, MSE = 6M) and for test question type (F (1,83^ = 5.52, £ < .002, MSB = 2M and 
a significant group-and-test-question-type interaction (f:.(3,83) = l'f.91, £ < .001). Table 2 
presents the means and SDs for each group. 

Table 2 ' 

1 

Group Mean Scores and Standard Deviations on Practice 
and Incidental TesV Questions— Experiment 2 







test Question Type 










Practice^ 


Incidental 








Group 


Me^n'' 


SD 


Meanb 


SD 


Total 




Read-only control 

Instructions 

Practice-questions 


7.33 
8.09 
9.70 


2.27 
1.78 


7.35 
. 9.88 
8.0'f 


^2.51 
1.51 
1.68 


l'f.68 
17.97 
17.7'> 




Practice-questions-plus- 
instructions 


10.97 


1.57 


8.50 


2.52 


19. ^7 






aQuestiorts were practice only for the practice-questions groups. 
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''Based on a maximum score of 13. 



The Newman-Keuls comparison of group means for the practice questions on the final 
test revealed that the performance of the practice-question and the practice-question- 
plus-instructions groups differed from that of the instructions and the read-only control 
groups (£ < .05). These data indicate that practice post-questions are more effective than 
instructions for material covered by the practice questions. This finding differs slightly 
from the results of the first experiment. A possible explanation for the superior 
performance of the practice-questions group on the practice questions of the final test in 
this experiment is that the paraphrased practice post-questions in the experimental 
materials were repeated on the final test. In the first experiment, all subjects saw the 
same words because the questions were taken verbatim from the text. 

The Newman-Keuls comparison of group means on the incidental questions of the; 
final test showed that the performance of the instructions and the practice-questions 
groups differed significantly from that of the read-only control group (jg < .05) but not 
from that of the practice-questions-plus-instructions group. These data also support the 
hypothesis that explicit instructions about how to study for comprehension can be as 
effective as practice ouestions in focusing attention and study behavior on incidental 
material. An interesting finding is that the performance of the practice-questions-plus- 
instructions group does not differ from that of the read-only control group on the 
incidental test questions. A possible explanation for this result is that the presence of the 
instructional reminder with the practice question may have caused the subjects to focus 
exclusively on the practice question and how it was paraphrased. Some support for this 
interpretation comes from the finding that the practice-questions-plus-instructions group 
spent the most time on the experimental materials. Further support for the overfocusing 
interpretation is provided by the correlation computed between performance on the 
practice and incidental test questions in each treatment group. The correlations for the 
read-only control, instructions, practice-questions, and practice-questions-plus-instruc- 
tions groups are .^8, .38, AS, and .21 respectively. All correlations were significant 
except that for the practice-questions-plus-instructions group, which indicates that this 
group's performance on the practice test items was not related to its performance on the 
incidental test items. Note that the performance of this group did differ from that of the 
read-only control group in experiment 1 and that the correlation for that group was 
significant. This overfocusing might have been avoided by informing the students that the 
test would cover more than just the practice questions. 



The third experiment examined the effects of explicit instructions and practice 
questions on learning a conceptual classification task from text. Classification tasks 
involve categorizing, sorting, or identifying objects, events, or things according to the 
category in which they belong based on their common characteristics. Classification 
tasks are higher-order or transfer tasks because they require subjects to deal with things 
or events not previously encountered. The intent of this Experiment was to determine 
whether explicit instructions could be written to enable students to learn a classification 
task effectively. 



Subjects and Materials 

Subjects, 279 Navy enlisted personnel, were randomly assigned to four groups: (l)a 
read-only control group, (2) a practice-questions group, (3) an instructions group, and W a 
practice-questions-plus-instructions group. The 657-word experimental passage was a 
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lesson on Navy radio call signs, which are used by radio stations to identify themselves 
(e.g., WABC or KCBA). The passage was divided into four sc-ctions, covering five types or 
classes of Navy call signs. 

A 2^-item short-answer test was developed that required subjects to classify four call 
sign instances from each of the f ive^call sign types and four call signs that were not valid 
Navy call signs. Twelve of the 2* call signs were used as practice questions. The practice * 
and incidental call sign examples were counterbalanced. • *\ 

Procedure " * ^ 

The subjects were randomly assigned to one of the experimental conditions. After 
they had worked through the appropriate materials at their own pace, they were required 
to classify the 2^ call signs based on their common characteristics. Treatment of the 
groups is described below. ^ 

1. The read-only control group read the passage and tpok the-test., 

2. Practice-questions group subjects were told that they would be required to 
answer questions after they had completed each of the four sections but they did not 
receive details as to the type of questions or study strategy. After they completed each 
section, they were given three call signs to classify and instructed to do so without 
referring back to the experimental materials. After they had completed the four 
sections, they took'the test, which included the 12 call signs they had classified previously 
as practice questions and 12 other call signs, used as incidental questions that they had 

'not classified'before. 

3. The instructions group was given explicit instructions; that is, a presumably 
effective study strategy for the present type of material, a definition, and examples of 
classification tasks. Subjects were told to study the experimental ^passage so that they 
could identify the critical features of the categories described therein and distinguish 
among them. After they had completed each of the four sections, they were given the 
following 5tudy reminder: "Remember, on the test you will be asked to classify a variety 
of call signs. There are several types of call signs that can be classified according to 
different characteristics . You should think about the characteristics of the type of call 
sign you just learned." When they had completed all sections, they took the test requiring ^ 
them to classify 2^ call signs, none of which they had classified before. 

^. The practice-questions-plus-instructions group received the same instructions 
provided the instructions group. After they had finished each section, they were given the 
practice questions (three call signs to classify), followed by the study reminder.. When 
they had finished all of the sections, they took the test requiring them to classify 2^ call 
signs, 12 of which they had classified as practice questions and 12 that they had not 
classified previously. * ^ 

Analyses 

A 2 (instructions ^%. J\o instructions) by 2 (practice questions vs. no practice 
questions) by 2 (practice tfest question vs. incidental test question) analysis of covariance 
with the first two factors between groups and the third factor within subjects was 
performed on the number correct on practice question/incidental question subtest of the 
final test. Subjects* Armed Forces Qualification Test (AFQT) scores were used as a 
covariate. 
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Results and Discussion 

The analysis of covariance for test scores showed a significant main effect only for 
practice questions (F (l,27'f)= i'f.89, £ < .001). Means and SDs for each group on the 
number of incident.. 1 and practice call signs classified correctly are presented in Table 3. 

Table 3 • 

Group Adjusted Mean Scores and Standard Deviations on ^ 
Practice and Incidental Test Questions— Experiment 3 







Test Question Type 






• 


Practice^ 


Incidental 






Group 


Meant" 


SD 


Mean^ 


SD 


Total 


Read-only control 


8.96 


2.89 


8.88 


3.00 


17.8'f 


Instructions 


9.38 


2.6^ 


9.36 


2.7H 


18. 7^* 


Practice-questions 


10.37 


2. Op 


10.3'f 


2.13 


20.71 


Practice-questions-plus- 






10.21 


2.'f3 


20.50 


instructions 


10.29 


2.38 



^Questions v^re practice only for the practice-questions groups. 
*^Based on a maximum score of 12. 

These data indicate that instructions describing a classification task and how b«t to 
process the information were not as ,ef f ective as were practice questions. Further, the 
performance of the instructions groUp did not differ from that of the read-only control 
group on the final test. It is interesting to note, however, that performance on the 
practice and on the incidental test questions did not differ for any of the groups. A 
possible explanation for this result is that good performance on categorization tasks does 
not depend on memory for specific items of information but, rather^, on memory for the 
defining characteristics of the category. 

Because these results conflict with the results of the first two experiments, 
structured protocols were taken on seven students while they read the materials given to 
the instructions group. The structured protocols consisted of questions designed to 
determine what affect the instructions had on classification task learning. Students were 
questioned after tKey read the instructions, when they were learning the materials, and 
after they completed the final test. In general, the questions concerned the students' 
awareness of what the test would be like and how best to study. After reading the 
instructions, only one student could accurately predict what the test would look like. 
During the learning period, two more students accurately described the final test. 
Finally, after completing the final test, five students reported that they would have 
studied differently had they seen the test questions in advance. These data help explain 
why the performance of the instructions group subjects did not differ from that of the 
read-only control group subjects. Apparently, subject? in these groups did not correctly 
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anticipate the final test, whereas subjects in the practice-questions groups saw actual test 
questions and presumably adjusted their study behavior accordingly. 

EXPERIMENT* 

For the fourth experiment, the instructions were refined to include (1) examples of 
classification tasks and (2) instructions on how classification tasks are tested and how to 
study for a classification task. 

Method " 1 

Subjects and Materials 

Subjects, 72 Navy enlisted personnel, were randomly assigned to four groups: a read- 
only control group, a practice-question group, an instructions group, and a practice- 
-questions-plus-instruction group. The 657-word experimental passage was a lesson on five 
classes of Navy radio call signs, which was divided into four sections, covering five types 
or classes of Navy call signs. 

A 2'f-item short-answer test was developed that required subjects to classify four call 
sign instances from each of the five call sign types and four call signs that were not valid 
Navy caU signs. Twelve of the 2'f call signs were used as practice questions. The question 
and incidental call sign examples were counterbalanced. 

Procedure . ■" 

After the subjects had worked through the appropriate experimental materials at 
their own pace, they were, given the final test. The groups were treated as follows: 

1. The read-only control group read the passage and took the test. 

2. After the practice-questions group subjects had completed each section, they 
were given three tall signs to classify and instructed to do so without referring back to 
the materials. After they had completed the four 'sections, they took the test, which 
included the 12 call signs they had classified previously as practice questions and 12 other 
call signs, used as incidental questions. 

3 The instructions group was given explicit instructions about what and how to 
study for,a classification task. Subjects were told to study so that they could identify the 
critidkl features of the categories described in the passage and distinguish- among them. 
(The instructions are provided in the*" appendix.) After they had completed each of the 
four sections, they were given the foUowing study reminder: "Remember, on the test you 
will be given several call signs. For each one, you will have to write the name of the type 
of call sign it is next to it. You should try to remember the characteristics of the type of 
call sign you just learned as well as the name of the type." When they had completed all 
sections, they took the test requiring them to classify 2'f call signs, none of which they 
had classified before. 

if. The practice-questions-plus-instructions group received the same instructions^ 
given the instructions group. After they had finished each section, they were given the 
practice questions (thi^ee call signs to classify), foUowed by the study reminder, when 
they hadb finished all the sections, they took the test requiring them to classify 2^ call 
signs, 12 of which they had classified as practice questions- and 12 that they had not 
classified before. 

* 21 . 
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Analyses 



1. Analyses of covariance were performed on the time required to complete the 
experimental materials and on the time required to complete the final test. 

2. A ^ (group) by 2 (practice test question vs. incidental test question) analysis of 
covariance with the first factor between subjects and the second factor within subjects 
was performed on the number correct on practice incidental questions on the final* test. ^ 
Subjects' AFQT score was used as a covariate. 

3. Group mean scores on practice incidental questions on the final test were ^ 
compared using a post hoc Newman-Keuls test. 

Results and Discussion 

A significant main effect for group was found for the analyses of covariance 
performed on the time to complete the experimental materials (F (3,67)= 3.75, £ < .01), 
on the time to complete the final test (F (3,67) = 3.53, £ < .01) and on the number correct 
^on practice/incidental questions on the final test (F (3,67) = ^.33, 2 < -01). Group means 
for required completion times and for test scores are presented in Tables ^ and 5 
respectively. 



Table ^ 

Group Mean Completion Times--Experiment ^ 





Completion Times (Min.) 


A- 


Group 


Experimental Materials 


Final Te§t 


Read-only group 


9.33 




Practice-questions 


12.72 


8.22 


Instructions 


11.05 


7.66 


Practice-guestions-plus-instructions 


12.77 


, 7.77 
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Table 5 

Group Adjusted Mean Scores and Standard Deviations on 
Practice and Incidental Test Qufestions--Experiment ^ 



Test Question Type 



Practice^ Incidental 



Group 


Meant> 


SD, 


Meant" 


SD . 


Total 


Read-only control 


6.28 


3.28 


6A5 


2.95 


12>3 


Instructions 


8.23^ 


3.79 


SA5 




16.68 


Practice-questions 




, 2.81 


SA7 


2.81 


16. 9^^ 


Practice-questions-plus- 








1.8^^ 


19.55 




instructions 




1.83 


9.61 
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^Questions were practice only for the practice-questions groups. 
'^Baoed on a maximum score of 12. 

It should be noted that the group mean scores in this study were generally higher than 
those in the third experiment, even though some of the groups received identical 
treatments. The reason for this is that the AFQT ocores for subjects in experiment 3 
were significantly Tower (p > .001) than those for subjects in the present experiment. 

As in experiment 3, group performance on practice questions and incidental questions 
on the final test did not differ. Again, an explanation for this result is that good 
performance on categorization tasks does hot depend on memory for specific items of 
information but, rather, on memory for the category name and it^ defining character- 
istics. 

The Newman-Keuls comparison of group means for both practice^ and' incidental 
questions oh the final test revealed that the performance of the read-only control group 
differed significantly (p < .05) from that of the other groups. These data indicate that the' 
instructions were as effective as practice questions in facilitafing learning. 

CONCLUSIONS 

In general, the instructional implications of these results are 'that students should be 
told explicitly what to expect on the test, as well as given practice questions. Although 
the strategy that would promote^ the most learning for factual material .would be to^ 
include practice questions for every item to be tested, this ^approach is usually^ 
irhpractical. The strategy of giving instructions plus practice' questions should allow 
students to concentrate more broadly- on all information relevant to the final test rather 
than focusing *on the portion of the content covered by the practice questions. However, 
experiment 2 revealed that providing the learner with instructions about paraphrased 
practice questions as well as the practice questions appeared to overfocus the student's 
/attention on the practice questions. This overfocusing is demonstrated by the poor 
performance of the practice-questions-plus-instructions group on the 'incidental test 
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items. A possible strategy for preventing overfocusing would be to develop more explicit 
instructions about the nature of the test and how to study .for it. 

The results of these studies indicate that instructions can be as effective and, in 
some cases.'more effective than practice questions in facilitating learni^jg from text. In 
addition, protocols and time measures indicate that instructions more direqtly con- 
trol/focus student processing and attention. Finally, a combination of instructions and 
practice questions are likely to be most effective. More research is needed^to investigate 
how instructions actually work to change or modify the student's studying.behaviors. This . 
can be done* by looking more closely at the proc|sses the student employs while reading . 
the ihstr'ictions and studying the materials. Knowing more about what the student is 
doing "and how the instructions affect his/her study behavior will enable instructional 
designers to be more precise iTi their prescriptions for writing Instructions and for , 
determining what specific kinds of instructions are most appropriate. 

* 

RECOMMENDATIONS 

1. It .is recommended that 'students be given both explicit instructions and questions 
during instruction. Questions and instructions should be given in both classroom and 
individualized instructional situations. 

2. It is recommended that instructions provided in the appendix be used with the 
appropriate instructional content. 
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EXPERIMENT 1 



1. Factual Instructions : 

Before reading the text, you will find a sample passage. Read through these 
paragraphs and then try to answer the questions that follow, to yourself, without going 
back to the passage. Turn the page to check your answers. The sample passage is to help 
you to determine the type of information you will be expected to remember in the 
experimental text that follows. 

2. Sample Passage ; 

Action is the basis of the initial knowledge acquired by the child. Through his 
actions, the infant can evolve general concepts of the things he encounters: His concept 
of a dog could be based on the feel of its hair and the funny noises it makes when pinched; 
his concept of a ball could be based on itsrbehavior when thrown. In the absence of 
- language, perceptual and cognitive structures can be built up out of the actions of his 
experiences. 

These early structures contain the sequences of actions taken by a child in dealing 
witlii his environment. The development of these schemes for action integrates sensory 
eveiits with motor movements in what is called a sensorimotor schema. The various 
sen'sorimdtor schemata learned by the child in his first 2 years, plus the knowledge he then 
hasfof the permanence and independence of objects and events, comprise the base upon 
which language 'Is established. 

Language develops w'^' nis framework of sensorimotor intelligence. Initially, the 
problem of learning a ' ^ua^e may mainly be. that of assigning linguistic labels-names- 
to the already existi..^ sensorimotor schemata. The child first begins to use language to 
label the things with which he is already familiar--.his toys, his family, the objects in his 
environment. He also labels his actions as h^ performs them. 

Answer these questions to yourself. Then turn the page and check your answers. 

Questions : . • , 

a. What is the basis of the iniii^nowledge acquired by the child? 

b. What is the schema for.i^jPPnng sensory events.with movement called? 

c. What is the initjai^foblem involved in.Iearning a language? 



Note . During the actual experiment, each part of the instructions was presented 
separately. 



27 

A- 1 



Answers ; 

a. Action. 

b. A sensorimotor schema. 

c. Assigning riames to objects. 

3 Reminder (Given after subjects read each section of experimental passage): ^' 

text. 
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EXPERIMENT 2 



1. Paraphrased Instructions : 

Before reading the text, you will fipd a warm-up passage and sample questions. The 
purpose of this warm-up passage is to illtistrate the kinds of questions you will be asked on 
the test. You will notice that the tjuestions ask for factual information that was 
contained in the sample passage. The Questions will not use the exact words that were 
used in the sample passage but, instead,,will use different words that describe the same 
ideas. Because the wording in the questions will be different, you should study so that you 
understand the facts presented in the pajsage. 

Turn the page and read the warm-up passage. When you have finished reading, 
answer the practice questions. Please nbte how the wording in the questions is different 
from the wording in the passage. 

2. Warm-up Passage ; 

In the Civil War, control of the sea was important and even decisive—and it was 
overwhelmingly in the hands of the North. For it, years the Union Navy was constantly 
occupied with the task of blockading more than 3000 miles of coasthne, rUnnmg down 
Southern commerce raiders, cooperating with the Army in capturing coastal strongholds, 
and opening the Mississippi and other waterways that led into the South. ' The South 
countered with commerce raiders, but the strangling effect of the Union blockade 
eventually took its toll. It crippled the finances of the Confederacy, shut out foodstuffs 
and munitions, and proved a major factor in deciding the outcome of the war. The 
country learned, or should have learned, from this war that a navy could not be quickly 
and readily improvised in an emergency. Even then, the days were past when merchant 
vessels could be converted rapidly into efficient men-of-war. 

When the Spanish-American War broke out, it was at once recognized that the 
conflict would be primarUy naval and would be won by the nation that secured control of 
the sea. The paper strength of the two navies was about equal, but the Spanish Navy was 
characterized by poor equipment, incompetence, and lack of trained personnel. Both at 
Manila Bay and Santiago the enemy's fleet was destroyed. Although Dewey's victory at 
Manila Bay had little material effect on the war, the destruction of Admiral Ctr^era's 
ships off Santiago, Cuba, established the Navy's command of the Caribbean. 

Answer the sample questions and then turn the page to checl< your answers. 
Questions 

a. During the war between the states, Yankee strategy for using their ships to the 
best advantage centered on • 

b. One of the lessons that was taught in the Civil War was that it took time to 

c. When the war between the U.S. and Spain began, it quickly became clear that the 
victor would be whichever country 



Note . During the actual experiment, each part of the instructions was presented 
separately. 
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Answers; 

a. Blockading the coastline. 

b. Build a Navy. 

c. Controlled the sea. 

Notice that the questions above asked for factual informatr6n that was contained in 
the- warm-up passage. The questions did not use the exact words that were in the sample 
passage, but, instead, used different words that described the same ideas. - 

3. Reminder (Given after subjects read each section of experimental passage): 

Remember that you will be tested on factual information found in the text. Because 
the Avording in the questions will be different, you should study so that you understand the 
main facts presented in the passage. 
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EXPERIMENT 4 



1. Classification Instructions : 

In this experiment, you will learn about a classification task. Here, we will tell you 
what classification tasks are, and how to study for them. 

Classifications refer to groups or categories of similar things. The things in each 
category are similar because they have certain special characteristcs in common. 
Classification tasks involve categorizing or sorting or identifying things according to 
which category they belong in. This is done on the basis of the special characteristics the 
things have. In order to classify things into the right category, you need to remember the 
special characteristics that define each category and the name of each category. 

Examples of Classification Tasks : 

The first example is political parties. If you know a person's political views on some 
issues, you should be able to classify the person as a Republican, Democrat, Independent, 
etc. To do this, you need to know what the names of the panties are and the 
characteristic political views associated with each party. For example. Republicans (the 
name) believe in free enterprise, less government, and a strong military (the character- 
istics). If you met a person who believed in free enterprise, a strong military, and less 
government, you would classify that person as a Republican. 

The second example of a classification task is ^Iso a familiar one. It has to do with 
automobiles and their makes or models. You can probably identify a car as a Buick, 
Chevrolet, Volkswagen, etc. You can do this because you know the names of different 
types of cars, and because you know what characteristics to look for to help you decide 
what make a car is. For example, if you saw a car with three holes on the front fenders, 
you would need to know that these holes are characteristic of a particular type of car, and 
the name of that type— a Buick. ^ 

Finally, a third example of a classification task is identifying types of wood. 
Carpenters are able to classify and sort various types of wood according to such special 
characteristics as grain, texture, and color. Carpenters know these characteristics for 
many types of wood, and they know the name of each type. For example, if we gave a 
carpenter a piece of reddish straight-grained, soft wood, he would know that these 
characteristics are typical of a particluar type of wood, and he knows the name of that 
type— Redwood. 

How are Classification tasks Tested? 

Classification tasks are usually tested by giving the student an example from some 
category, and asking the student what the name of the category is. To do this, the 
student needs to know the characteristics of different categories, and he needs to 
remember the names of each of the categories. For example, to test whether someone 



Notes. 

1. Instructions given here were revised from those used in experiment 3. 

2. During the actual experiment, each part of the instructions was .given separately. 
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cCd identify cars, we wouid S've a -ie^ of pictures of ca^^^^^^ f^^/sTw^ 
Ihe name of each car. test whe'her ^m^^^^^^^^ V^^^^ ,yp, „f «o<x) 

a A typical classification test has directions Bke this: 

..Here is a list of things. Writ* the name of each one in the space provided next 
to each one." 

b. Here is a test for the task of classifying cars: 

r • <-ar<: Write the name or model of 

"Directions: Here is a series of pictures of cars. Write tne 

each car next to its picture, 
(picture!) 



(picture 2) 
(picture 3) 
etc." 



c Here is a test fo^the task of identifying poUtical parties: 
' ..Directions: Here is a list of different peoples- political views. Next to each 
one, write what party you think the person belongs to. 

(0 Fred believes in less government, a strong military, and free enterprise. 



Fred is a 



(2) Alice believes in big government, consumer protection, more welfare 
instead of defense spending. 

Alice is a 

etc." 

Hnu/ to stud y for Cla ssification Tasks 

usually, a piece of instruction ""J, Scars' mig"s^n,~^^ 
particular category. For example, f^'=°°l<J,''°"' ^ front fenders." This sentence tells 

&\!m1^^^^^^^^^ 
hembtr'trnr^e: ^^Ir^^^^ -t. even if you do remember 

the characteristics. 

2. Reminder (Given after subjects read each section of exp^erimental passage): 

Remember, on the test you wUl be given several call signs. For -h^one^you w,U 
^elerti;: JSs£* *ryp:'of"^l.% you iust learned as well as the ^ 
of the type. ^ 
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